The invention relates to processing multiview video.
Multiview Video Coding (MVC) relates to compression of video sequences (e.g., a sequence of images or “pictures”) that are typically acquired by respective cameras. The video sequences or “views” can be encoded according to a standard such as MPEG. A picture in a video sequence can represent a full video frame or a field of a video frame. A slice is an independently coded portion of a picture that includes some or all of the macroblocks in the picture, and a macroblock includes blocks of picture elements (or “pixels”).
The video sequences can be encoded as a multiview video sequence according to the H.264/AVC codec technology, and many developers are conducting research into amendment of standards to accomodate multiview video sequences.
Three profiles for supporting specific functions are prescribed in the current H.264 standard. The term “profile” indicates the standardization of technical components for use in the video encoding/decoding algorithms. In other words, the profile is the set of technical components prescribed for decoding a bitstream of a compressed sequence, and may be considered to be a sub-standard. The above-mentioned three profiles are a baseline profile, a main profile, and an extended profile. A variety of functions for the encoder and the decoder have been defined in the H.264 standard, such that the encoder and the decoder can be compatible with the baseline profile, the main profile, and the extended profile respectively.
The bitstream for the H.264/AVC standard is structured according to a Video Coding Layer (VCL) for processing the moving-image coding (i.e., the sequence coding), and a Network Abstraction Layer (NAL) associated with a subsystem capable of transmitting/storing encoded information. The output data of the encoding process is VCL data, and is mapped into NAL units before it is transmitted or stored. Each NAL unit includes a Raw Byte Sequence Payload (RBSP) corresponding to either compressed video data or header information.
The NAL unit includes a NAL header and a RBSP. The NAL header includes flag information (e.g., nal_ref_idc) and identification (ID) information (e.g., nal_unit_type). The flag information “nal_ref_idc” indicates the presence or absence of a slice used as a reference picture of the NAL unit. The ID information “nal_unit_type” indicates the type of the NAL unit. The RBSP stores compressed original data. An RBSP trailing bit can be added to the last part of the RBSP, such that the length of the RBSP can be represented by a multiple of 8 bits.
There are a variety of the NAL units, for example, an Instantaneous Decoding Refresh (IDR) picture, a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), and Supplemental Enhancement Information (SEI), etc.
The standard has generally defined a target product using various profiles and levels, such that the target product can be implemented with appropriate costs. The decoder satisfies a predetermined constraint at a corresponding profile and level.
The profile and the level are able to indicate a function or parameter of the decoder, such that they indicate which compressed images can be handled by the decoder. Specific information indicating which one of multiple profiles corresponds to the bitstream can be identified by profile ID information. The profile ID information “profile_idc” provides a flag for identifying a profile associated with the bitstream. The H.264/AVC standard includes three profile identifiers (IDs). If the profile ID information “profile_idc” is set to “66”, the bitstream is based on the baseline profile. If the profile ID information “profile_idc” is set to “77”, the bitstream is based on the main profile. If the profile ID information “profile_idc” is set to “88”, the bitstream is based on the extended profile. The above-mentioned “profile_idc” information may be contained in the SPS (Sequence Parameter Set), for example.